ShowTable of Contents
Introduction
IBM® Lotus® Domino® Database Attachment & Object Service (DAOS), also known as Attachment Consolidation, is a new feature in version 8.5 whereby all file attachments are stored in a separate repository on the server and are no longer stored in individual NSF’s. Multiple copies of attachments are only stored once.
Enabling DAOS
To administer DAOS, an administrator (and only an administrator) performs the following steps:
Enable DAOS on the server
1. Ensure Transaction Logging is enabled in the Server document, as shown in figure 1, and be sure to set a value for the Maximum Log Space field.
Figure 1. Transaction Logging tab
2. Enable DAOS in the DAOS tab of the Server document (see figure 2).
Figure 2. DAOS tab
3. Add Create_R85_databases=1 to the server Notes.ini file. This parameter configures the server to create new databases using the 8.5 ODS (51). By default, all new databases in 8.5 are created with ODS 43, unless this parameter is enabled.
4. Restart the server and verify that DAOS is now enabled on the server by entering the command, “show server”. The result should be “<<DAOS: Enabled ”.
Enabling DAOS results in the following changes:
- There is now a DAOS folder located beneath the Domino\Data folder called Domino\Data\DAOS. This is the object store repository.
- The DAOS Catalog (DAOScat.nsf) is created.
Enable IBM Lotus Notes® databases
Enabling DAOS in the Server document does not configure Notes databases to use DAOS. You must decide which databases should participate in DAOS and enable it for those particular databases.
To enable DAOS for only one database, or to enable it for one database at a time:
- Select Database --- Application --- Properties, and select the Advanced tab.
- Enable the option “Use Domino Attachment and Object Store” (see figure 3).
Figure 3. Advanced tab of Database Properties
To enable multiple databases at the same time:
- Go to the Domino Admin Client Files tab and select the databases you want to have enabled.
- Select Tools --- Databases --- Advanced Properties (see figure 4).
Figure 4. Advanced Properties
3. Enable the “Use Domino Attachment and Object Service” option.
After enabling either of the above methods, all newly created attachments that meet the minimum size requirement will be managed by DAOS and will no longer be stored in the NSF’s.
Best practices and troubleshooting FAQs
Best practices
Backing up. The importance of having a backup process in place cannot be stressed enough, and failing to do this is a huge exposure.
The backup process is the same for DAOS as for general Notes usage; the process has not changed with the introduction of DAOS. Note, however, that with a standard database, only the NSF needs to be backed up, whereas with DAOS, the NLO files must also be backed up. The NSF files should be backup up first and then the NLO files.
Without a backup, you could lose data (or at least the ability to restore an older NSF and have the associated NLO files available).
Domino version. Even though DAOS was introduced in Domino 8.5, it is very important that you use at least v8.5.1FP3. Using v8.5.2 plus the latest fixpack---or even better v8.5.3 plus the latest fixpack--- is the best option as there are many improvements in Domino 8.5.2 and 8.5.3 for DAOS. The improvements are aimed at helping to keep the DAOS catalog in a “Synchronized” state, as well as reducing the impact of a resync, if one is needed.
DAOS debug. The following parameters should be added to the server Notes.ini.
- console_log_enabled=1
- debug_threadid=1
- debug_daos_statechange=1
- daos_logging=c:\notesdata\IBM_TECHNICAL_SUPPORT\daos.log ERROR WARNING RESYNC
NOTE: The “ERROR WARNING” text should always be there. You then need to add something specific to it, such as “RESYNC”, so you will be looking for debug output on any errors that refer to a resync.
A server restart is required as the daos_logging parameter requires a cold start of the API to pick up.
Prune. The PRUNE command comes into play only if an NLO has no reference, for example, if associated with a mailfile. As long as an NLO file has a reference (.nsf), it will stay around forever, even with prune=0. Prune can be run only if the DAOS catalog state is Synchronized. Also, before running prune, you should back up all NLO's.
Transaction logging. Disabling transaction logging will effectively disable DAOS and cause all (new) attachments to be written inline in the NSF.
Duplicate NLOs. Duplicate NLOs are intermediate objects that most likely were used as part of a conversion between different encoding/compression styles. So verify that different compression types and different encoding methods are not being used. Duplicate NLOs can also be created if you are using mail journaling.
Recreating the DAOS Catalog. If you need to do this:
- Ensure you have a backup procedure in place and then delete the catalog.
- Restart the server; the catalog will be automatically recreated upon server restart.
- After Lotus Domino has recreated the catalog, use the Tell DAOSMgr Resync command to re-synchronize DAOS references.
Offline resync with Microsoft® Windows®. We are all familiar with running an online resync, but sometimes an offline resync is much more beneficial. To do this, from the data directory run the command ..\ndaosmgr resync from the Windows command prompt.
Offline resync with Linux®. With Linux the command is daosmgr resync. Log in as the UNIX Notes user, cd to the data directory, and then <path to daosmgr> resync.
Clustering. Since DAOS is server based, each server must be DAOS enabled in a cluster environment, and each server in the cluster set must have its own data store. Sharing is not allowed, so it's not a good idea to restore a DAOS store from one server with the DAOS store on its cluster mate.
Remember that DAOS is server-specific, so if you want DAOS enabled on all servers in a cluster, you must configure it on each server. It is also acceptable to have one or more servers in the cluster not using DAOS.
Replication. When a DAOS-enabled database on a DAOS-enabled server replicates with another DAOS-enabled server, the full attachments replicate over to the other server. Once there, they are treated as DAOS attachments on the receiving server also. When you create a new replica of a database, the full attachments will replicate.
If the new replica is on a DAOS-enabled server, then the attachments will automatically be treated as DAOS attachments on the receiving server also. If the destination replica is not on a DAOS-enabled server, the full attachments will replicate anyway but will be stored within the NSF’s.
FAQ's
Q1. When is it appropriate to delete the DAOS Catalog?
A. Many users are confused as to whether this is safe to do. Is it safe? Sure. Is it a good idea? Not usually. The only time you should delete the daoscat database are:
- when performing a disaster recovery of an entire system, since daoscat.nsf and daos.cfg should not be backed up or restored, and it's vital that those files have an accurate accounting of what really resides on the system, or
- if it's damaged beyond repair. Think along the lines of deleting the transaction logs, or even Names.nsf. It should be a last-ditch option.
Deleting daoscat.nsf effectively resets the “clock” on the entries on the deferred deletion list, so they are all dated when the first resync was run after it was deleted. Deleting daoscat.nsf is rarely a good thing to do. As with Names.nsf, you wouldn't delete it, unless it was corrupted beyond repair.
Q2. Is it ever appropriate to run maintenance on the DAOS Catalog?
A. No, you should never run maintenance on daoscat.nsf, for two reasons. First, it should not need it, because there are no documents or views in it. Secondly, since it's always open and in use, there is never a time that it can be successfully opened with exclusive access needed for maintenance. There are some rare problem cases that can be corrected with “compact”, but there is a specific procedure for doing that, and it requires extreme caution.
Q3. In a clustered environment, why are there just 28 directories on one node and 34 on the other one? Is that normal? They are both configured the same.
A. This disparity is not something to worry about. The exact order of filling/creating/using the directories depends on what order the traffic arrived at the server and what operations were performed on the NSF files in the meantime. As long as the total number of NLO files is (relatively) close, it's fine.
Q4. Is it OK to run a “fixup -j -D”?
A. No, this could very well delete even more data and make it unrecoverable. You should never run fixup -D, unless you are positive you have a good copy of the NSF somewhere with which to replicate and a good backup in place.
The -N switch is used to ensure that corrupt documents are not purged, so use –N along with – j –D and also ensure you have a backup and a good replica in place.
Q5. How long should it take for a resync to run?
A. Resync will either (1) see there's nothing to do because the catalog is in a SYNC state and exit immediately (a matter of seconds), (2) see there is work to do and run for up to 5 minutes, or (3) error out on something immediately due to a problem (a few seconds).
The exact times will vary, depending on the amount of data on each configuration.
Q6. Is it possible to switch the Domino Server ID of a system running DAOS?
A. No, because by default the DAOS repository is encrypted with the Server ID. This means that the DAOS repository can only be moved to new hardware (or restored from backup) to the same Server ID file.
If DAOS encryption is on, currently the only way to migrate to a new server is either to create new replicas on the new server and move the data that way, or use “compact -c -daos off” to re-integrate all the attachments, and then OS-copy the NSF files to the new server.
If not using encrypted files, then if a user has DAOS_ENCRYPT_NLO=0 in the server's Notes.ini and that has been enabled since before their first .NLO file was creates, then yes, you can switch IDs.
Q7. Can you take an NLO in the repository and determine in which database it is saved?
A. For disk performance reasons, DAOS does not keep "backpointers" to which NSFs have references, so there is no way to say "given this NLO name, which databases uses it". The only thing that is supported is listing which NLOs are used for a specific database, using “tell daosmgr listnlo -o dbname.lst all dbname.nsf.”
Q8. Why are all NLOs saved only to the 001 directory?
A. This is a known issue, documented in SPR# PMAO83ZRE4, and customers should open a Support request for access to the Hot Fix for their specific configuration.
Q9. Should daos.cfg and daoscat.nsf be backed up?
A. No, the daos.cfg and daoscat.nsf files should not be backed up. If these files become corrupted, they can be safely deleted while the Domino server is not running. They will be created upon startup automatically, and a DAOS resync will then completely re-populate them from scratch. However, refer to Q1 above about deleting the DAOS catalog.
Conclusion
In summary, it is crucial that you ensure DAOS is set up correctly from the outset. The most important thing when working with DAOS is that you have a good backup in place. Not having this is a huge exposure resulting in possibly being unable to recover from issues such as data loss.
Tell us what you think
Please visit this link to take a one-question survey about this article:
Resources
developerWorks® Lotus Notes and Domino product page:
http://www.ibm.com/developerworks/lotus/products/notesdomino/
developerWorks white paper, “IBM Lotus Domino Attachment and Object Service: Why optimizing transaction logging matters;”
http://www.ibm.com/developerworks/lotus/documentation/daosoptimizing/
Notes and Domino forum:
http://www-10.lotus.com/ldd/nd85forum.nsfAbout the author
Paula Power works on the Server Core team in Level 2 support at IBM's Dublin Support Center, specializing in the area of database and DAOS. You can reach her at
Paula_Power@ie.ibm.com.